checkout: Provide useful error with checkout -H and incompat mode

checkout: Provide useful error with checkout -H and incompat mode

Previously we'd assert and dump core if one used `checkout -H` without
`-U` on a bare-user repo, because we'd hit the bare-user symlink case.

Rework the code to handle this, and add tests. I hit this when I was going to
suggest to someone to use `-H` to ensure they were getting hardlinks.

Closes: #779
Approved by: jlebon